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Abstract 

Background: Mental health problems and risk behaviours among young people are of great public health concern. 
Consequently, within the VII Framework Programme, the European Commission funded the Saving and 
Empowering Young Lives in Europe (SEYLE) project. This Randomized Controlled Trial (RCT) was conducted in 
eleven European countries, with Sweden as the coordinating centre, and was designed to identify an effective way 
to promote mental health and reduce suicidality and risk taking behaviours among adolescents. 

Objective: To describe the methodological and field procedures in the SEYLE RCT among adolescents, as well as to 
present the main characteristics of the recruited sample. 

Methods: Analyses were conducted to determine: 1) representativeness of study sites compared to respective 
national data; 2) response rate of schools and pupils, drop-out rates from baseline to 3 and 12 month follow-up, 
3) comparability of samples among the four Intervention Arms; 4) properties of the standard scales employed: Beck 
Depression Inventory, Second Edition (BDI-II), Zung Self-Rating Anxiety Scale (Z-SAS), Strengths and Difficulties 
Questionnaire (SDQ), World Health Organization Weil-Being Scale (WHO-5). 

Results: Participants at baseline comprised 12,395 adolescents (M/F: 5,529/6,799; mean age=14.9±0.9) from Austria, 
Estonia, France, Germany, Hungary, Ireland, Israel, Italy, Romania, Slovenia and Spain. At the 3 and 12 months follow 
up, participation rates were 87.3% and 79.4%, respectively. Demographic characteristics of participating sites were 
found to be reasonably representative of their respective national population. Overall response rate of schools was 
67.8%. All scales utilised in the study had good to very good internal reliability, as measured by Cronbach's alpha 
(BDI-II: 0.864; Z-SAS: 0.805; SDO: 0.740; WHO-5: 0.799). 
(Continued on next page) 
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(Continued from previous page) 

Conclusions: SEYLE achieved its objective of recruiting a large representative sample of adolescents within 
participating European countries. Analysis of SEYLE data will shed light on the effectiveness of important 
interventions aimed at improving adolescent mental health and well-being, reducing risk-taking and self-destructive 
behaviour and preventing suicidality. 

Trial registration: US National Institute of Health (NIH) clinical trial registry (NCr00906620) and the German Clinical 
Trials Register (DRKS00000214). 

Keywords: SEYLE, Mental Health Promotion, Suicide prevention. Promotion, Well-being, Adolescents, Schools, RCF, 
Intervention, ProfScreen, QPR, Awareness 



Background 

In the transition from childhood to adulthood adoles- 
cents make lifestyle choices and initiate patterns of 
behaviour that affect both their current and future well- 
being and health [1-6]. Many adverse health behaviours 
emerge in adolescence and track into adulthood, with 
increasing consequences for negative and sometimes 
long-lasting outcomes. Given the importance of this 
transitional period, it is essential to systematically assess 
the mental health and well-being of adolescents and 
young adults, and to implement and evaluate interven- 
tions for at-risk individuals. Several large studies have 
been carried out, mostly in the US, to gather informa- 
tion on both healthy and risk behaviours as well as 
psychiatric symptoms, based on robust methodologies 
[7-11]. Other studies analysed the effects of interven- 
tions to promote mental health and prevent suicide 
among adolescents [12-14]. 

However, to the best of our knowledge, no previous 
study compared the effectiveness of interventions based 
on different approaches with a Randomized Controlled 
Trial (RCT). The Saving and Empowering Young Lives 
in Europe (SEYLE) project was designed with this in 
mind. 

SEYLE, supported by the European Union Seventh 
Framework Program (FP7), (Grant agreement number 
HEALTH-F2-2009-22309), is an RCT to evaluate school- 
based preventive interventions of risk-taking and self- 
destructive behaviours in eleven European countries, 
including: Austria, Estonia, France, Germany, Hungary, 
Ireland, Israel,^ Italy, Romania, Slovenia and Spain, with 
the National Centre for Suicide Research and Prevention 
of Mental Ill-Health (NASP) at Karolinska Institutet (KI) 
in Sweden responsible for the scientific coordination of 
the project. The Child Psychiatric Epidemiology Group at 
Columbia University and New York State Psychiatric 
Institute served as methodological experts. SEYLE is 
registered in both the US National Institute of Health 
(NIH) clinical trial registry (NCT00906620) and the 
German Clinical Trials Register (DRKS00000214). The full 
protocol of the study has been previously published [15]. 
The key objectives of the study were: (i) to collect 



assessment data on a cohort of European adolescents, 
including demographic information, psychopathology, life- 
styles, values and risk-behaviours, in order to produce an 
epidemiological database on the general health status of 
European adolescents; (ii) to evaluate three types of 
school-based interventions in comparison to a minimal 
intervention control group. The three active interventions 
included (1) teacher training, (2) increasing adolescents' 
awareness about mental health, and (3) professional 
screening of adolescents for mental health problems and 
risk behaviours. Teachers were trained through the gate- 
keeper program: Question, Persuade and Refer, developed 
in the US by the QPR Institute [16]. Pupils were trained 
through a standardized awareness-increasing program 
[15,17] designed to promote knowledge of mental health, 
healthy lifestyles and behaviours among adolescents. A 
professional screening program performed by psychiatrists 
and psychologists was specifically designed for the SEYLE 
study. All pupils were screened with a questionnaire and, 
if responses exceeded a predetermined cut-off score for 
depression, anxiety, phobia, alcoholism, substance abuse, 
non-suicidal self-injury (NSSI) or suicidality, pupils were 
interviewed and then referred for professional treatment if 
necessary. More details about the SEYLE interventions 
have been previously published [15]. 

The objectives of this article are to describe: 1) the 
study sites; 2) the main methodological issues employed; 
3) the characteristics of the recruited sample including 
its representativeness; and 4) the internal reliability of the 
psychometric scales utilized for evaluation of the 
outcomes of the RCT: the Zung Self-Rating Anxiety Scale 
(Z-SAS, [18]), the Beck Depression Inventory, Second 
Edition (BDI-II, [19]), the World Health Organization 
Well- Being Scale (WHO-5 [20], and the Strengths and 
Difficulties Questionnaire (SDQ, [21]. 

Methods 

Study sites 

SEYLE had one study site in each of the eleven 
European countries described above. At each site at least 
one study catchment area, reasonably consistent with an 
administratively established geographic area was selected. 
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The selected catchment areas in each country are 
described in Table 1. To meaningfully interpret the poten- 
tial representativeness, key parameters, such as mean age, 
number of immigrants, population density, net income 
and gender proportion for each site were compared to the 
corresponding national data. Data at the national and local 
levels were extracted from Eurostat [22] and collected for 
each participant site. Effect sizes of mean age and number 
of immigrants at the country and study site levels were 
calculated for each country according to Cohen's d, mea- 
sured as small (d=0.3), medium (d=0.5) and large (d=0.8). 
Differences in gender distribution among 15-year olds at 
the country and study site levels were evaluated with a test 
of proportions. Population density and net income were 
compared between each country's national data and the 
respective study site. 

School and participant selection 

At each site, eligible schools were randomly selected to 
participate in SEYLE. A list containing all available 
schools was generated at each site, and the schools were 
categorized as large or small and randomized into one of 
the four study arms for possible inclusion according to a 
randomized order. Simple randomization was used as a 
method of randomization of schools through a random 
number generator. Schools were categorized as small if 
they had less than or equal to the median number of 
students in all schools in the study area/region; and large 
if they had greater than the median number of students 
in all schools in the study area/region. Schools were 
considered eligible if they were public, contained at least 
forty 15-year-old pupils, had more than two teachers for 
pupils 15 years of age and no more than 60% of the 
pupils were of the same gender. These inclusion criteria 
were selected to allow for the recruitment of a 



comparable sample of schools and pupils across study 
sites, in spite of differences in sociocultural factors and 
in the organization of the educational system. However, 
a few exceptions were made in the case of sociocultural 
particularities of a specific country's education system 
and applying the exact same criteria would increase 
selection bias instead of reducing it across sites. In 
particular in Ireland, single gender schools were allowed 
to participate in pairs with a single gender school of the 
opposite sex and of similar size. In Germany, due to the 
unique design of the school system, a sample of schools 
in the three categories of German high schools were 
selected and randomized separately. National and/or 
regional school authorities were contacted and informed 
about the project in general terms in order to get 
approval, which was obtained in all participating coun- 
tries. The representatives of SEYLE at each study site 
then met with the school principals in the respective 
areas to describe the intervention of the Arm to which 
their school had been randomized and to explain the 
general objectives and procedures of that Arm. Each 
school was selected to participate in one Arm only and 
no information was disclosed about the interventions to 
be performed in other Arms of the RCT. On the basis of 
general information about SEYLE objectives and specific 
information about the specific intervention Arm into 
which the school was randomized, the school could 
accept or refuse to join. When a school refused to 
participate, the next school randomized in the same 
category was approached to replace it. It is important to 
note that schools were replaced only with other schools 
that were already in the randomization list. This proce- 
dure was designed to generate a balanced number of 
large and small schools in each intervention Arm, to 
minimize bias and increase the validity of the results. 



Table 1 Demographics of SEYLE study sites, according to Eurostat^ 



Country 


Study site 


Population 


IVlean age^ 


% females 


Pop. density 


Income 

(net,EURO per year) " 


Eurostat area 


Austria 


Tirol 


704,472 


39.9 


51.1 


56.1 


22,192 


Trol 


Estonia 


Tallinn 


524,938 




54.0 


122 


7,905 


Pohja-Eesti 


France 


Lorraine 


2,348,384 


40.3 


512 


99.7 


19,182 


Lorraine 


Germany 


Heidelberg 


10,749,506 


41.8 


50.8 


300.7 


24719 


Baden-Wurttemberg 


Hungary 


Budapest 


2,925,500 


40.7 


53.3 


1794 


8,735 


Kozep-Magyarorszag 


Ireland 


Cork and Kerry 


548,700 




50.7 


53.5 




Ireland South West 


Italy 


Region Molise 


320,795 


44.0 


514 


734 


14,315 


Mollse 


Romania 


Cluj and Maramures counties 


2,721,468 


38.7 


513 


77.8 


2,755 


Romania Nord-Vest 


Slovenia 


Osrednjeslovenska, Podravska 
and Obalno-kraska region 


965,200 




50.8 


1554 


9,889 


Podravska, 
Osrednjeslovenska 
and Obalno-kraska 


Spain 


Oviedo, Gijon and Aviles 


1,058,923 


45.3 


52.2 


101.9 


14767 


Principado de Asturlas 



^Eurostat. Statistics database. European Union; 2010. Available from: http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/. 
^Data from Israel is not included as the study site was the whole country. 
^Not available for study sites in Estonia, Ireland and Slovenia. 
^Not available for study site in Ireland. 
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Within each school, all classes with a majority of 15 year 
olds were approached for participant recruitment, with a 
minimum of two schools per Arm. This procedure was 
repeated until a minimum of 250 students were re- 
cruited in a each Arm. Prior to requesting consent from 
the parents and assent from the pupils, general informa- 
tion about the SEYLE study and details about the 
specific Arm they were invited to participate in, was 
provided. Not all pupils for whom parental consent and 
adolescent assent were obtained actually participated, as 
some students were absent from school on the day the 
questionnaire was administered. Consent rates were 
calculated as the percentage of approached pupils for 
whom parental consent and pupil's assent were both 
given. Participation rates were calculated as the per- 
centage of assented pupils with parental consent who 
actually took part in the baseline questionnaire. In order 
to evaluate the impact of the consent rates of schools 
and pupils on the external validity of the collected data; 
school size, in terms of number of attending pupils, was 
compared between participating and non-participating 
schools. Moreover, gender proportion of pupils with and 
without consent were compared. Drop-out rates were 
calculated as the number of pupils assessed at baseline 
who did not participate at the first (3-months) and/or 
second (12-months) follow-up. Sociodemographic varia- 
bles obtained at baseline and average scores on the 
scales employed, were used to evaluate differences 
between Arms. 

Instruments and interventions 

A full description of assessment instruments and inter- 
ventions was previously published [15]. 

Standardization of methodology 

Each SEYLE site used the same methodology in an effort 
to obtain comparable study results. Homogenous meth- 
odology was achieved through two different means. First, 
a detailed procedures manual (328 pages) was developed, 
containing information regarding every aspect of the 
study implementation, including school selection, re- 
cruitment, randomization, clinical backup, ethical issues, 
translation procedures and methods of cultural adapta- 
tion, detailed descriptions of each intervention and 
intervention time-lines, as well as the baseline and 
follow-up questionnaires. Second, uniform training pro- 
cedures were conducted. All site leaders were initially 
trained centrally, in Stockholm. Site leaders then 
conducted local training for their own teams that 
included a minimum of 27 hours of group work, with at 
least 4 hours devoted to each intervention. To ensure 
study fidelity to the methodology, a series of monitoring 
site visits were carried out by representatives of the 
coordinating centre (NASP), together with the two 



consultants from Columbia University visiting each site. 
The site visits took place to overlap with training by site 
leaders and consisted of two-day consultations with local 
staff involved in the SEYLE project. Present at the site 
visit were the Intervention Arm coordinators, as well as 
the site leader (Table 2). During the site visit, local staff 
were required to present their understanding of the 
study and its procedures, as described in the manual, as 
well as the requirements of conducting each of the 
interventions. Site visits also provided an opportunity to 
correct any misunderstandings and to provide additio- 
nal training, if necessary, to assure adherence to the 
protocol. 

Cross-site collaboration was an important study 
objective and was facilitated prior to data collection, 
throughout implementation and up until study comple- 
tion. For example, most sites assumed primary responsi- 
bility for one major study requirement, called a Work 
Package (e.g., translations, cultural adaptation, quality 
control, ethical requirements, data management, etc.) 
and collaborated with each of the other sites concerning 
this specific topic. Some of the major collaborative 
efforts are described below. 

Quality control procedures 

As part of the SEYLE project, a method for quality 
control was developed and implemented. A series of 
questionnaires were sent to intervention coordinators in 
each country in order to ensure that all preparatory 
procedures were correctly conducted and that the inter- 
ventions implemented at each site were faithful to the 
initial intervention models of SEYLE. 

Analysis of these data allowed for assessment of the 
degree of discrepancy between different sites and be- 
tween implementation in each site compared with the 
SEYLE model, as well as the effect variations had on the 
projects overall results and conclusions. Three quality 
control assessment tools were used - I. A questionnaire 
administered during site visits to assess the preparedness 
for intervention implementation; II. A pre-intervention 
questionnaire focused on questionnaire coding and on 
specific requirements to be carried out prior to and 
immediately after the intervention, and; III. A post- 
intervention questionnaire focused on the implementation 
of each intervention. Analyses of these questionnaires 
showed very small differences between the sites in the 
implementation process and did not identify any major 
modification in the implementation of the interventions 
in any site. 

Translation and cultural adaptation 

The Hungarian site coordinated the translation pro- 
cesses in collaboration with the site-specific translation 
coordinators and the coordinating centre in Sweden 



Table 2 SEYLE study key personnel 



Executive committee 



Coordinator and Project Leader 


Danuta Wasserman 


National Centre for Suicide Research and Prevention of Mental Ill-Health (NASP) at Karolinska Institutet (Kl), Stockholm, 
Sweden 


Deputy Coordinator 


Marco Sarchiapone 


Department of Health Sciences, University of Molise, Campobasso, Italy 




Project Manager and Assistant Project Leader 


Vladimir Carii 


National Centre for Suicide Research and Prevention of Mental Ill-Health (NASP) at Karolinska Institutet (Kl), Stockholm, 
Sweden 


Consultants for Methodology 


Christina Hoven 


Department of Child and Adolescent Psychiatry, Columbia University-New York State Psychiatric Institute, New York US 




Camilla Wasserman 














Intervention arm coordinators 


QPR 


Vladimir CarIi 


National Centre for Suicide Research and Prevention of Mental Ill-Health (NASP) at Karolinska Institutet (Kl), Stockholm, 
Sweden 


ProfScreen 


Romuald Brunner / Michael 
Kaess 


Clinic of Child and Adolescent Psychiatry, Centre of Psychosocial Medicine, University of Heidelberg, Heidelberg, Germany 


Awareness 


Camilla Wasserman 


Department of Child and Adolescent Psychiatry, Columbia University-New York State Psychiatric Institute, New York, US 


Minimal Intervention/ Control 


Marco Sarchiapone 


Department of Health Sciences, University of Molise, Campobasso, Italy 




Study sites 


Country 


Site leader 


Site 


Arm coordinators 




Translation 


Workpackage leadership 






coordinator 


QPR 


ProfScreen 


Awareness 


coordinator 




Austria 


C. Haring 


P. Olesky 


C Pajek 


P. Olesky 


C. Haring 






Estonia 


A. Varnik 


R. Soonets 


M. Sisask 


L. Heidmets 


R. Soonets 


K. Vailing 


P.Varnik (Data management) 


France 


JP. Kahn 


F. Guillemin 


A. Tubiana 


H. Vann 


B. Bucki 


JP Kahn 




Germany 


R. Brunner 


M. Kaess 


N. Schonbach 


M. Kaess 


K. Klug 


M. Kaess 


R. Brunner (Intervention Coordinator) 


Hungary 


J. Balazs 


J. Balazs 


M. Balint 


G. Meszaros 


L. Farkas 


J. Balazs 


J. Balazs (Translation procedures) 


Ireland 


P. Corcoran 


H. Keeley 


C McAuliffe 


F. Elahi/ 


J. McCarthy 


H. Keeley 


L-A. Burke (Analysis of cost-effectiveness) 










P. Cotter 








Israel 


A. Apter 


D. Feldman 


C Burzstein 


S. Hen-Gal 


Y. Apter 


Y. Apter 


D. Feldman (Quality control) 


Italy 


M. Sarchiapone 


G. Nicolais 


V. CarIi 


F. Basilico 


M. losue 


M. Sarchiapone 


M. losue (Dissemination) 


Romania 


D. Cozman 


B. Nemes 


0. Dobrescu 


B. Nemes 


D. Herta 


B. Nemes 


D. Cozman (Materials for Schools) 


Slovenia 


V. Postuvan 


V. Postuvan 


U. Mars 


T. Podlogar/ 


V. Postuvan/ 


V. Postuvan 












V Kosir 


J. Ziberna 






Spain 


J. Bobes 


P. Saiz 


E. Diaz-Mesa 


M. Garrido 


S. Al-Halabi 


P. Saiz 


P. Saiz (Cultural adaptation) 


Sweden 


D. Wasserman 


V. CarIi 










D. Wasserman (Project coordination and data 
analysis) 



Table 2 SEYLE study key personnel (Continued) 



Administrative assistants 



Tony Durkee 



National Center for Suicide Research and Prevention of IVlental Ill-Health (NASP) at Karolinska Institutet (Kl) 



Brigit Frisen-Andersson 



Pierre Bodin 



Anna Lundgren 



External Ethical Advisor 



Stella Reither-Theil 



-University Hospital Basel, Psychiatric Clinics of the University of Basel, IBMB, University of Basel, Switzerland 
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(NASP). Translation coordinators oversaw site-specific 
translations, back translations and pilot interviews of all 
SEYLE materials. All materials were forward and back 
translated in each participating language. German was 
used in both Germany and Austria but was translated by 
the German site. All SEYLE materials (instruments & 
intervention Arm packages) were developed originally in 
English and then translated into the following languages: 
Gaelic (Irish), German, Estonian, French, Hebrew, 
Hungarian, Italian, Romanian, Slovenian and Spanish. In 
order to confirm the quality of the translations, staff 
from each site reviewed all back translations, evaluated 
reports on the respective pilot interviews and provided 
feedback to the Hungarian site. The site translation coor- 
dinators also implemented cultural adaptation: primarily 
concerning local linguistic phenomena and expressions. 
Focus groups were then conducted at each site to provide 
feedback on the cultural adaptation resulting from the 
pilot testing. In the case of ambiguity, consultation with a 
cultural linguistic advisor was sought. Based on these pro- 
cedures, culturally adjusted language replaced the original 
in the final versions. A report concerning language issues, 
including possible ambiguity was sent to the coordinating 
centre for resolution, when necessary. 

The scales used in the SEYLE questionnaires were 
included in the officially translated and validated version, 
when available, in the respective language, e.g., the SDQ 
[21], the WHO-5 etc. [15]. If the scale was not avaOable in 
the required language it was translated (and back- 
translated) for SEYLE, using the same procedure as for the 
other study materials. Internal reliability for all scales used 
in SEYLE was assessed through Cronbach's alpha [23] . 

Data entry and data quality control 

Data were collected on paper questionnaires, with the 
exception of the Austrian site, where data were collected 
electronically for direct data entry. In Germany, data 
collection forms were scanned for automatic data entry 
while in the other nine countries manual independent 
double data entry procedures were followed. In those 
countries information was entered twice using the Statis- 
tical Software Package SPSS 17.0. The Estonian site, 
which was responsible for the data management proce- 
dures, provided continuous oversight to other centers 
and promptly responded to any queries arising during 
the data entry process. A two-stage data cleaning and 
quality control procedure was performed to guarantee 
clean and reliable data. The first stage of quality control 
was performed locally, based on the two data files gener- 
ated through double data entry. These files were com- 
pared and inconsistencies were resolved by checking the 
paper material. Based on this corrective action an accur- 
ate data file was generated. The second stage of quality 
control was performed centrally at the Estonian site, by 



double-checking each local dataset, attempting to detect 
other errors, such as incompleteness (missing values), 
inconsistencies (incorrectly followed skip-outs), irregu- 
larities (numbers inserted in text variables), and out of 
range data (i.e. very large number of siblings or sexual 
partners). The results of these control procedures gener- 
ated a list of queries that was sent for resolution to the 
specific site. After finalizing quality control procedures, 
the Estonian site pooled the data into one database for 
all respondents for each wave (i.e., baseline, 3-month 
and 12-month follow-up). Pooled databases for each 
wave were then merged into one longitudinal pooled 
database. 

Ethics and emergency Issues 

Ethical issues were discussed with an independent eth- 
ical advisor from Basel University in Switzerland. Each 
site obtained permission from the local ethics committee 
to implement the SEYLE study in their respective coun- 
try. According to guidelines from the local ethics com- 
mittees, after thorough examination of the SEYLE study 
objectives and procedures, decisions were made locally 
to obtain consent through an opt-in method (parents 
had to sign a consent form if they allowed their child to 
participate) or an opt-out method (parents had to sign a 
refusal form if they did not want their child to partici- 
pate). Study subjects were then recruited into the study 
accordingly, after obtaining the required informed con- 
sents from parents and assent from pupils [24]. 

A specific procedure to identify and immediately assist 
emergency cases with acute suicidality was implemented 
at each site in all four Arms. A minimum set of require- 
ments regarding the identification of emergency cases 
was followed by each center. However each centre had 
the opportunity to reinforce the ethical requirements, 
according to the indications of the local Ethics Commit- 
tee. Emergency cases were identified through responses 
to two specific questions in the questionnaire: those who 
reported moderate or severe suicidal ideation in the 
previous two weeks, or those who reported attempting 
suicide in the previous two weeks. Subjects identified as 
emergency cases were followed-up by local SEYLE 
personnel until successful referral to the local healthcare 
system. However, since it was not possible to follow up 
individuals while in treatment due to confidentiality, it is 
not known if the clinical intervention had impact on the 
collected data. However, all emergency cases were 
allowed to participate in the active interventions and in 
the control arm. Therefore, these pupils are included in 
the total data set. 

Subsequent to conducting SEYLE, an interdisciplinary 
workshop'' was held in the Psychiatric Clinics of the 
University Basel, to analyse ethical issues, especially con- 
fidentiality towards minors involved in SEYLE Study. 
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Additionally, relevant codes and guidelines for guidance 
in mental health research with minors were analysed. 
While unresolved questions may remain, such as 
whether and when confidentiality might or should be 
overridden in cases of emergency [25], within the SEYLE 
study, all problems of confidentiality were determined to 
have been handled according to the indications of the 
local ethical committees and the local laws and regula- 
tions of the participating countries. 

Results 

Consent, participation and drop-out rates 

Response rates for SEYLE are reported in terms of 
consent and participation rates for schools and pupils. 

Schools 

If a school refused to participate, no pupil in that school 
was approached. The school randomization with 
replacement methodology, however, required refusal 
schools to be replaced by the next school of the same 
size on the school randomization list. Response rates for 
SEYLE schools, by country, are reported in Table 3. A 
total of 264 schools were approached for participation. 
Of these, 179 schools accepted, with an overall response 
rate of 67.8%. However, the school response rate was 
72.0% when Israel, the only study site to have a low re- 
sponse rate (37.5%), was excluded. School size, measured 
by the total number of students in the school, of partici- 
pating and non-participating schools did not differ in 
any country, with the exception of Slovenia, where par- 
ticipating schools were smaller than non-participating 
schools. 



Table 3 SEYLE study school response rates, including 
number randomized, approached and participated, by 
country 



Country 


Randomized 
schools 


Approached 
schools 


Accepted to 
participate 


Response 
rate* 


Austria 


22 


22 


15 


68.2% 


Estonia 


23 


23 


19 


82.6% 


France 


25 


25 


20 


80.0% 


Germany 


100 


41 


26 


634% 


Hungary 


23 


19 


15 


78.9% 


reland 


24 


24 


17 


70.8% 


srael 


32 


32 


12 


37.5% 


Italy 


18 


18 


14 


77.8% 


Romania 


27 


19 


16 


84.2% 


Slovenia 


32 


20 


13 


65.0% 


Spain 


23 


21 


12 


571% 


Total 


349 


264 


179 


67.8% 



* percentage of approached schools that accepted to participate in the 
SEYLE study. 



Pupils 

Rates of pupils' consent have been calculated for the 
eight countries (Estonia, Germany, Hungary, Ireland, 
Israel, Italy, Romania and Spain) that used similar ethical 
procedures in collecting pupils consent. The overall rate 
of consent in these eight countries was 76% (10,665 pu- 
pils with consent out of 14,086 approached). In the other 
three countries (Austria, France and Slovenia), extended 
procedures for collection of the informed consent were 
imposed by the local ethics committees (i.e., multiple 
forms to be signed; pupil could be enrolled only if both 
parents signed the form, etc.). This resulted in a consent 
rate of 23% (3,452 pupils with consent out of 14,803 
approached) in these three countries. When combining 
these three countries with the other eight the overall 
rate of consent decreased to 49% (14,117 pupils with 
consent out of 28,889 approached). Of the total 14,117 
pupils whose parents gave consent, 12,395 participated 
in SEYLE, yielding a participation rate of 87.8%. Gender 
proportion of consented and non-consented pupils did 
not significantly differ in any country with the exception 
of France and Slovenia, where more girls were present 
among participating pupils. Information regarding 
gender proportion of non-participating pupils was not 
available in Ireland and Germany. 

Overall, in the 3 months follow-up assessment, 10,823 
pupils participated and 9,846 pupils participated at 12 
months. The overall 12-month drop-out rate from 
baseline was 20.6%, including a 12.7% at 3 months. The 
drop-out rate did not differ significantly between coun- 
tries and ranged between a minimum of 18.6% in the 
Control Arm and a maximum of 23.2% in the Awareness 
Arm. The differences in the socio-demographic and 
psychopathological characteristics at baseline, between 
those who participated in all waves of data collection 
and those who dropped out, did not differ significantly 
between Arms (Table 4). All schools remained actively 
involved through the three waves of data collections 
with no school drop-out. Differences in the socio- 
demographic and psychopathological characteristics 
between those who participated in all waves of data 
collection and those who did drop out did not differ 
significantly between Arms. 

Sample characteristics 
Age and gender 

The age and gender distribution of the sample, stratified 
by country, is shown in Table 5. Gender distribution of 
the 12,395 participating pupils was 6,799 females and 
5,529 males (67 with missing gender data); the mean age 
was 14.91±0.90 (83 with missing age data). The largest 
sample was recruited in Germany (n=1444). Austria 
(n=960) was the only country that did not reach the 
target of 1000 pupils at baseline. Eight study sites 
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Table 4 Participation in SEYLE according to Intervention Arm, including baseline, 3 and 12 month follow-up and drop 
out rates, by gender 



Intervention arm 


Gender 


Baseline number 

V^cilUci /Oj 


3 Month follow-up 

ilUIIIUci ^^cllUci /O) 


3 Month 

UlU|J UUL IdLtr'^ \ '^i 


12 Month follow-up 

II ^^cllUci /O) 


12 Month 

Uiup (JUL idLc V 


QPR 


Ma es 




1 1 c;q M -2 1 ) 


1 2 5 


1 04^ iA'^ V\ 

1 U'H-J J. 3 j 


21 2 




1 CI 1 laicj 




1 J 1 J \J\J.DJ 


106 


1 pu. 1 j 


20 2 




Rr\tn npnHprc 


3036 


2689 


1 1 4 


241 0 


20 6 


AVVd 1 Cl Icib 


Ma es 


1 m (AA 


1 1 C\f^ (A'^ A) 


1 8 1 


Q7Q (Al 1 \ 


27 5 




Females 


1 /QA Q\ 

\ 004 ij4.yj 




1 A 1 
\ 4. 1 




1 o o 

1 y.y 




Both genders 


3032 


2551 


15.9 


2329 


232 


ProfScreen 


Males 


1301 (42.4) 


1158 (42.1) 


11.0 


1024 (41.7) 


213 




Females 


1752 (57.1) 


1583 (57.5) 


9.7 


1423 (58.0) 


18.8 




Both genders 


3070 


2752 


10.4 


2455 


20.0 


Minimal Intervention 


Males 


1554 (47.7) 


1 323 (46.7) 


14.9 


1 239 (46.7) 


20.3 




Females 


1689 (51.9) 


1494 (52.8) 


11.6 


1403 (52.9) 


16.9 




Both genders 


3257 


2831 


13.1 


2652 


18.6 


Total 




12395 


10823 


12.7 


9846 


20.6 



*From baseline. 



recruited more females than males (Table 5). A larger 
number of males were recruited only in Ireland (54.7%), 
Israel (81.4%) and Spain (51.7%). In an analysis of repre- 
sentativeness, on the basis of Eurostat data [22], very 
small effect sizes were found concerning variations in 
the mean age between study sites and the respective 
country. Cohen's d effect size also remained lower than 
0.3 for the total sample when stratifying the analysis by 
gender. The largest effect size for both genders was 
found in Spain (d=0.205). For all other countries, the 
effect size of age was below 0.1. Differences in the 
proportion of 15-year old males and females and the 
respective country's data were not statistically significant 

Table 5 SEYLE pupil participation by Country, according 
to mean age and gender 



Country Pupils Gender 





N= 


Mean age (SD) 


Male 




Female 










n" 


% 


n" 


% 


Austria 


960 


15.1 (0.8) 


350 


36.8 


602 


63.2 


Estonia 


1,036 


14.2 (0.5) 


477 


46.0 


560 


54.0 


France 


1,000 


152 (0.8) 


319 


31.7 


688 


68.3 


Germany 


1,444 


14.7 (0.8) 


692 


47.9 


752 


52.1 


Hungary 


1,009 


15.1 (0.8) 


415 


41.1 


594 


58.9 


reland 


1,091 


137 (0.7) 


600 


54.7 


496 


45.3 


srael 


1,256 


15.9 (0.8) 


1,023 


814 


233 


18.6 


Italy 


1,189 


153 (0.7) 


381 


32.0 


811 


68.0 


Romania 


1,139 


15.0 (0.4) 


395 


34.6 


745 


654 


Slovenia 


1,165 


152 (0.7) 


347 


29.7 


823 


70.3 


Spain 


1,023 


14.5 (0.7) 


530 


51.7 


496 


48.3 


Total 


12,312 


14.9 (0.9) 


5,529 


44.8 


6,799 


552 



^83 pupils with missing age data have been excluded. 
"^67 pupils with missing gender data have been excluded. 



at any site. Analysis of representativeness was not 
conducted in Israel as the study site was of the entire 
country. 

Population density 

Population density at the study sites was higher than in 
the respective country in Estonia, Germany, Hungary, 
Ireland and Spain. Population density was lower at the 
study site in Austria, France, Italy, Romania and 
Slovenia. 

Income 

The difference in net income per inhabitant between 
each country and the respective study site was below 
10%, with the exception of Estonia (-1-17%), Germany 
{+15%), Hungary (-h42%) and Italy (-24%). 

Immigrants 

The proportion of immigrants in each study site popula- 
tion was not significantly different from the proportion 
of immigrants in the respective country in all countries 
with the exception of Italy (-5%), Slovenia (-8%), and 
Spain (-10%). 

Unemployment rates 

In no country were unemployment rates at the study site 
significantly different than in the respective country as a 
whole. 

Therefore, based on these key parameters, the pupils 
participating in the SEYLE study can be considered 
reasonably representative of their respective country. 

Additionally, the main socio-demographic indicators 
such as age, gender, belonging to a single parent house- 
hold, belonging to a religious denomination and parental 
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unemployment did not significantly differ between 
Arms. 

Internal reliability of psychometric scales 

The internal reliability of each scale was assessed sepa- 
rately for each country. The results are reported in 
Table 6. The internal reliability for the Z-SAS [18], the 
BDI-II [19], the WHO-5 [20] and the SDQ [21] was high 
or very high in most countries. 

Discussion 

A large landmark intervention study, using RCT design, 
necessitates an article dedicated to the description of 
methodological issues and their complexity, which 
requires more space than is usually allowed in the 
Methods section of an ordinary article in the majority of 
scientific journals. This paper describes the complex 
methodological issues in the SEYLE study, which will 
allow for adequate interpretation of study's results gener- 
ated over-time, as well as appropriate replication and 
development of the study in the future. 

SEYLE is a multi-site RCT of interventions to promote 
mental health and prevent risk behaviours and suicide in 
European schools. Very few RCTs have been conducted 
on youth mental health and most of them have focused 
on a single intervention or treatment method on a small 
sample or at only one site or within only one country, or 
alternatively with a clinical population [26-35]. SEYLE 
was designed to evaluate three different active interven- 
tion methods that respectively empower students, 
teachers and professionals, compared to controls, to 

Table 6 Cronbach alpha of scales administered in the 



SEYLE study, by country (n=1 2,395) 



Country 


Z-SAS 


BDI-ir 


WHO-5 


SDQ 


Austria 


.826^ 


.871 


.752 


.876 


Estonia 


.803^ 


.849^ 


.760^ 


.839^ 


France 


.844^ 


.869 


.810 


.824 


Germany 


.829^ 


.875 


.746 


.789 


Hungary 


.811^ 


.835^ 


.796 


.730 


reland 


.821 


.872 


.804 


.848 


srael 


.783^ 


.890^ 


.907^ 


.863^ 


Italy 


.638^ 


.806 


.765 


.717 


Romania 


.811^ 


.864^ 


.748^ 


.806^ 


Slovenia 


.855^ 


.867 


.734 


.716 


Spain 


.773^ 


.872 


.773 


.612 


Total 


.805 


.864 


.799 


.740 



Z-SAS, Zung Self-Rating Anxiety Scale; BDI-II, Beck Depression Inventory, 

Second Edition; WHO-5, World Health Organization Weil-Being Scale; SDQ, Strengths 

and Difficulties Questionnaire; 

^Item 21 of the BDI-II was not administered. 

^Item 6 not administered in Israel and not included in the assessment of 

internal reliability. 

^translated by the SEYLE study. 



identify early mental health problems and risk beha- 
viours, while facilitating appropriate referral to the 
healthcare system. The interventions were performed on 
a large sample (N=12,395) in eleven sites, located in 
eleven different European countries. Extensive proce- 
dures were implemented in order to guarantee a homo- 
geneous methodology across sites, including high quality 
forward and back-translations of manuals, instruments, 
standardized interventions, as well as cultural adaptation 
for each participating country and expert review of all 
ethical issues related to the investigation. 

The SEYLE project achieved the sampling size object- 
ive of enrolling at least 1,000 participating pupils at each 
site (except Austria; n=960), for a total sample of 
N= 12,395 school-based adolescents. Female participants 
(55.2%) exceeded the number of male participants. It 
may be hypothesized that girls are more interested and/or 
collaborative in participating in a study dealing with 
psychological issues than males, leading to a higher 
participation. However, in most countries, there were no 
significant differences between the gender proportion in 
the school and the gender proportion in our sample. 

Analysis of representativeness indicates that the study 
sites are reasonably representative of their respective 
countries, thus allowing for in-country and between- 
country comparisons. The overall response rate of 
schools was high (67.8%). Only Israel had a low response 
rate of schools (37.5%). Without Israel, response rate of 
schools was 72%. It can be hypothesized that the low 
response rate of schools in Israel was attributed to the 
nearly uniform attitude of school principals' against 
using school time for additional non-educational acti- 
vities, in view of the many such activities already taking 
place. Israel, along with Cork, Ireland and Oviedo, Spain, 
were the only sites where a majority of the participating 
adolescents were male. The Cork study site had the 
lowest pupil participation rate (64.6%), which can 
possibly be attributed to factors outside the scope of the 
study, as an environmental emergency affecting the 
region (flooding) at the time of the SEYLE study, thus 
preventing many pupils attending school when the base- 
line questionnaire was administered. However, overall 
pupil participation rates in SEYLE were high and thus 
assure adequate external validity of the collected data. 

Drop-out rates at follow-up were low: 20.6% at 
12-months, including 12.7% at three months follow-up, 
indicating broad acceptance of the interventions and 
questionnaires by both schools and pupils. Drop-out 
rates did not vary significantly among countries. Impor- 
tantly, the study methodology required that the school 
randomization include all eligible schools in the area. 
This allowed for comparability of study Arms within and 
across sites. The main demographic indicators at base- 
line, such as mean age, family structure and parental 
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unemployment did not differ significantly between the 
Active interventions and the Control Arm. 

Internal reliability of each scale administered in each 
country also provides reassuring results. Cronbach's 
alpha values were measured for both instruments 
translated for the purposes of SEYLE, as well as for 
instruments already available officially, in the respective 
languages. As reported in Table 6, Cronbach's alpha 
values were quite homogenous across countries with 
very small variations and can be considered good or very 
good for all administered scales. The lowest internal 
reliability was reported for the SDQ (alpha=0.740). This 
result is more than acceptable and in agreement with 
previous studies [36]. 

Strengths 

The major strength of the SEYLE RCT is its application 
of a robust and homogenous methodology applied 
across eleven study sites in eleven different countries, 
selected to provide a broad geographical representation 
of Europe. Due to extensive collaboration across sites 
through Work Packages, that required cross-site cooper- 
ation of all participating sites throughout the study, 
uniform adherence to the study methodology was 
assured. Moreover, the standardized translation method- 
ology and cultural adaptation allowed for the fine-tuning 
of interventions to be responsive to local cultural 
contexts, thus ensuring that the project was meaningful 
and useful data were collected at each site. Another 
major strength of the project is the inclusion of a control 
group and the selection of outcome measures, which are 
related to mental health and wellness, as well as risk 
behaviours, thus allowing for the study outcomes to be 
associated with three distinct interventions. Finally, the 
SEYLE interventions are able to be tested on a com- 
bined, large sample of European adolescents, generating 
the first such findings from a large-scale RCT of adoles- 
cent well-being in Europe, providing an important 
cohort that can be followed over time. 

Limitations 

In any large-scale multi-site study using a complex 
methodology, securing sufficient funding is always an 
important challenge. In the case of SEYLE, there were 
two major limitations due to funding: namely, the 
funding duration precluded a long follow-up after the 
intervention ended. It would have been of greater value 
to identify the long-term effects of the SEYLE interven- 
tions by having a longer follow-up, as many preventive 
effects may only be observed after a longer time post- 
intervention. In fact, a five-year instead of a three-year 
timetable for SEYLE would probably have allowed for 
more knowledge to be gained regarding the study's out- 
comes. In SEYLE, one site per country was chosen for 



study participation. Sufficient funds to allow the inclusion 
of more than one site per country would significantly have 
improved representation of the urban and rural areas and 
therefore understanding of different populations. More- 
over, the analysis of representativeness of the recruited 
sample in relation to the respective country was limited by 
the availability of sociodemographic indicators in Eurostat 
at the local level (NUTS2). It was not possible to directly 
compare the SEYLE data and the same indicators at the 
country level because these were not available for the 
adolescent population or were collected with different 
methodologies, ultimately being incompatible. 

Consent rates of schools and pupils varied across 
countries. The consent rates of pupils were very good in 
eight countries and lower in the three countries where 
extended consent procedures were imposed by the local 
ethics committees. However, it has been reported that 
response rates between 30% and 70% are, at most, only 
weakly associated with bias [37]. Available indicators 
such as school size did not differ significantly between 
participating and non-participating schools with the 
exception of Slovenia, where more small schools partici- 
pated in the study. The study was necessarily performed 
during school hours and consequently there was limited 
opportunity to collect other than questionnaire data 
regarding pupil's behaviour. This school-based approach 
necessarily required a very limited number of outcome 
measures. Another limitation is that all data were 
collected through self-report questionnaires. 

Conclusions 

The SEYLE RCT study was successful in recruiting 
a reasonably representative sample of over 12,000 
European school-based adolescents. The study is unique 
in its' robust and uniform methodology applied 
across eleven sites, including a large number of socio- 
demographic, lifestyle and mental health outcomes, 
allowing for evaluation of the effects of three Interven- 
tion Arms compared to a Control Arm. Several impor- 
tant indicators, such as response and participation rates, 
differences between Arms and reliability of scales show 
very good validity of the collected data and ensure that 
the selected outcome measures are reliable and useful 
for carrying out school-based identification of at-risk 
adolescents. The SEYLE database contains up-to-date 
information about lifestyles and mental health problems 
of European adolescents and will be of great benefit for 
mental health professionals, policy makers and other 
stakeholders throughout the European Union. 

Endnotes 

"Israel belongs to the WHO European Region and is 
eligible to receive funding under the European VII 
Framework Programme. 
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